<!DOCTYPE html>
<html lang="en"><head>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1"><!-- Begin Jekyll SEO tag v2.7.1 -->
<title>吾托帮消息系统传输Java对象_笔记 | userzb’s blog</title>
<meta name="generator" content="Jekyll v4.1.1" />
<meta property="og:title" content="吾托帮消息系统传输Java对象_笔记" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="背景说明" />
<meta property="og:description" content="背景说明" />
<link rel="canonical" href="https://userzb.github.io/notes/2021/01/04/%E5%90%BE%E6%89%98%E5%B8%AE%E6%B6%88%E6%81%AF%E7%B3%BB%E7%BB%9F%E4%BC%A0%E8%BE%93Java%E5%AF%B9%E8%B1%A1_%E7%AC%94%E8%AE%B0.html" />
<meta property="og:url" content="https://userzb.github.io/notes/2021/01/04/%E5%90%BE%E6%89%98%E5%B8%AE%E6%B6%88%E6%81%AF%E7%B3%BB%E7%BB%9F%E4%BC%A0%E8%BE%93Java%E5%AF%B9%E8%B1%A1_%E7%AC%94%E8%AE%B0.html" />
<meta property="og:site_name" content="userzb’s blog" />
<meta property="og:type" content="article" />
<meta property="article:published_time" content="2021-01-04T00:00:00+08:00" />
<meta name="twitter:card" content="summary" />
<meta property="twitter:title" content="吾托帮消息系统传输Java对象_笔记" />
<script type="application/ld+json">
{"dateModified":"2021-01-04T00:00:00+08:00","datePublished":"2021-01-04T00:00:00+08:00","@type":"BlogPosting","mainEntityOfPage":{"@type":"WebPage","@id":"https://userzb.github.io/notes/2021/01/04/%E5%90%BE%E6%89%98%E5%B8%AE%E6%B6%88%E6%81%AF%E7%B3%BB%E7%BB%9F%E4%BC%A0%E8%BE%93Java%E5%AF%B9%E8%B1%A1_%E7%AC%94%E8%AE%B0.html"},"url":"https://userzb.github.io/notes/2021/01/04/%E5%90%BE%E6%89%98%E5%B8%AE%E6%B6%88%E6%81%AF%E7%B3%BB%E7%BB%9F%E4%BC%A0%E8%BE%93Java%E5%AF%B9%E8%B1%A1_%E7%AC%94%E8%AE%B0.html","description":"背景说明","headline":"吾托帮消息系统传输Java对象_笔记","@context":"https://schema.org"}</script>
<!-- End Jekyll SEO tag -->
<link rel="stylesheet" href="/assets/main.css"><link type="application/atom+xml" rel="alternate" href="https://userzb.github.io/feed.xml" title="userzb's blog" /></head>
<body><header class="site-header" role="banner">

  <div class="wrapper"><a class="site-title" rel="author" href="/">userzb&#39;s blog</a><nav class="site-nav">
        <input type="checkbox" id="nav-trigger" class="nav-trigger" />
        <label for="nav-trigger">
          <span class="menu-icon">
            <svg viewBox="0 0 18 15" width="18px" height="15px">
              <path d="M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.032C17.335,0,18,0.665,18,1.484L18,1.484z M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.032C17.335,6.031,18,6.696,18,7.516L18,7.516z M18,13.516C18,14.335,17.335,15,16.516,15H1.484 C0.665,15,0,14.335,0,13.516l0,0c0-0.82,0.665-1.483,1.484-1.483h15.032C17.335,12.031,18,12.695,18,13.516L18,13.516z"/>
            </svg>
          </span>
        </label>

        <div class="trigger"></div>
      </nav></div>
</header>
<main class="page-content" aria-label="Content">
      <div class="wrapper">
        <article class="post h-entry" itemscope itemtype="http://schema.org/BlogPosting">

  <header class="post-header">
    <h1 class="post-title p-name" itemprop="name headline">吾托帮消息系统传输Java对象_笔记</h1>
    <p class="post-meta">
      <time class="dt-published" datetime="2021-01-04T00:00:00+08:00" itemprop="datePublished">Jan 4, 2021
      </time></p>
  </header>

  <div class="post-content e-content" itemprop="articleBody">
    <h2 id="背景说明">背景说明</h2>

<p>最近，我需要通过吾托帮消息系统来传输Java对象。</p>

<p>那何为传输Java对象呢？其实就是说传输Java对象要做什么，我的总结是：</p>

<p>（1）将Java对象序列化为指定类型的消息，并push到吾托帮消息系统中。</p>

<p>（2）从吾托帮消息系统中pull回指定类型的消息，并反序列化为Java对象。</p>

<p>吾托帮消息系统的底层是基于RabbitMQ，调用接口是基于Servlet进行再次封装的，传输协议是HTTP，传输的消息类型为字符串。所以问题我们已经明确清楚了，接下来就是编码实现。而实现过程中也发现了很多问题，我把它们一一列出来。</p>

<h2 id="存在问题">存在问题</h2>

<h3 id="编码问题">编码问题</h3>

<p>因为传输的消息类型为字符串，而序列化后的Java对象为byte[]。所以这里就有一个问题，如何将byte[]转换为String。具体的做法有：</p>

<p>(1) 基于String的byte[]和String的转换。直接通过new String(byte[])来获取String。然后通过getByte来获取byte[]。</p>

<p>但是这里有个坑，执行下面的测试代码，我们会发现结果为false。其主要原因是UTF-8的encode和decode会导致结果不一致，我们需要我们要定义ISO-8859-1编码或其它会使结果一致的编码。但因为这些编码会产生一些奇奇怪怪的字符（字符集过多），而这些奇奇怪怪的字符在HTTP请求中可能会存在截断(如’#’)或丢失(如’+’，会被识别为’ ‘)的情况，所以在传输Java对象这种场景中，我们不会使用基于String的方法来实现byte[]和String之间的转换。</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>List&lt;Integer&gt; tx = new ArrayList&lt;&gt;();
byte[] byteArray = SerializeUtil.serialize(tx);
byte[] byteArray1 =  new String(byteArray).getBytes();
System.out.println(Arrays.equals(byteArray,byteArray1));

</code></pre></div></div>

<p>(2) 基于Base64的byte[]和String的转换。这是业界的普遍做法。</p>

<p>Base64编码和解码已经在JDK中提供了。其主要分为三类，参考资料[2]：</p>

<ul>
  <li>基础，输出被映射到一组字符A-Za-z0-9+/，编码不添加任何行标，输出的解码仅支持A-Za-z0-9+/。</li>
  <li>URL，输出映射到一组字符A-Za-z0-9-_，输出是URL和文件。</li>
  <li>MIME，输出隐射到MIME友好格式。输出每行不超过76字符，并且使用’\r’并跟随’\n’作为分割。编码输出最后没有行分割。</li>
</ul>

<p>具体可参考Base64源码</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/**
 * This array is a lookup table that translates 6-bit positive integer
 * index values into their "Base64 Alphabet" equivalents as specified
 * in "Table 1: The Base64 Alphabet" of RFC 2045 (and RFC 4648).
 */
private static final char[] toBase64 = {
    'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',
    'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
    'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm',
    'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
    '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '+', '/'
};

/**
 * It's the lookup table for "URL and Filename safe Base64" as specified
 * in Table 2 of the RFC 4648, with the '+' and '/' changed to '-' and
 * '_'. This table is used when BASE64_URL is specified.
 */
private static final char[] toBase64URL = {
    'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',
    'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',
    'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm',
    'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',
    '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '-', '_'
};
</code></pre></div></div>

<p>因为传输协议是基于HTTP的，基础Base64存在+等字符，’+’在请求时会被识别为’ ‘字符，所以这里我们的选型为URL的Base64编码。</p>

<p>使用方法</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>String message = "";
byte[] byteArray = new byte[0];;

Base64.Encoder urlEncoder = Base64.getUrlEncoder();
message = urlEncoder.encodeToString(byteArray);

Base64.Decoder urlDecoder = Base64.getUrlDecoder();
byteArray = urlDecoder.decode(message);
</code></pre></div></div>

<h3 id="序列化后的对象太大">序列化后的对象太大</h3>

<p>调用接口时，报了以下错误。其实问题很快就能定位到，其根本原因是GET请求的长度有限，当然我们也可以通过压缩的方法来实现基于GET请求的传输。</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&lt;head&gt;
	&lt;title&gt;414 Request-URI Too Large&lt;/title&gt;
&lt;/head&gt;
&lt;body&gt;
	&lt;center&gt;&lt;h1&gt;414 Request-URI Too Large&lt;/h1&gt;&lt;/center&gt;&lt;hr&gt;
	&lt;center&gt;nginx&lt;/center&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre></div></div>

<p>解决方案：</p>

<p>(1) 将GET请求转换为POST请求，并将传入的参数从URL移到 form中。由于项目中使用的是RestTemplate，所以具体做法参考[1]。</p>

<p>(2) 在序列化的时候，使用GZIPOutputStream和GZIPInputStream来压缩。其具体实现见下：</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public static byte[] serialize(Object obj) throws IOException {
    try(
        ByteArrayOutputStream byteArrayOut = new ByteArrayOutputStream();
        //1.define the GZIPOutputStream
        GZIPOutputStream gzout = new GZIPOutputStream(byteArrayOut,true);
        ObjectOutputStream out = new ObjectOutputStream(gzout);
    ){
        out.writeObject(obj);
        out.flush();//2.flush
        result = byteArrayOut.toByteArray();
	}
	return result;
}
	
public static Object deserialize(byte[] byteArray) throws ClassNotFoundException, IOException {
    Object result = null;
    try(
        ByteArrayInputStream byteArrayIn = new ByteArrayInputStream(byteArray);
        //3.define the GZIPInputStream
        GZIPInputStream gzin = new GZIPInputStream(byteArrayIn,byteArray.length);
        ObjectInputStream in = new ObjectInputStream(gzin);
    ){
    	result = in.readObject();
    }
    return result;
}
</code></pre></div></div>

<h2 id="注意">注意</h2>

<p>(1) FastJson JSONObject的toString()方法，会将byte[]类型转换为Base64编码(注意是标准的)，所以’+’还需要自己二次编码。</p>

<p>参考资料：</p>

<p>[1] RestTemplate提交表单数据的三种方法 https://blog.csdn.net/yiifaa/article/details/77939282</p>

<p>[2] Java Base64 https://www.runoob.com/java/java8-base64.html</p>

<p>[3] Base64编码详解及其变种(解决加号在URL变空格问题） https://www.cnblogs.com/lifesting/archive/2012/07/12/2587923.html</p>

  </div><a class="u-url" href="/notes/2021/01/04/%E5%90%BE%E6%89%98%E5%B8%AE%E6%B6%88%E6%81%AF%E7%B3%BB%E7%BB%9F%E4%BC%A0%E8%BE%93Java%E5%AF%B9%E8%B1%A1_%E7%AC%94%E8%AE%B0.html" hidden></a>
</article>

      </div>
    </main><footer class="site-footer h-card">
  <data class="u-url" href="/"></data>

  <div class="wrapper">

    <h2 class="footer-heading">userzb&#39;s blog</h2>

    <div class="footer-col-wrapper">
      <div class="footer-col footer-col-1">
        <ul class="contact-list">
          <li class="p-name">userzb&#39;s blog</li><li><a class="u-email" href="mailto:zhibinz@126.com">zhibinz@126.com</a></li></ul>
      </div>

      <div class="footer-col footer-col-2"><ul class="social-media-list"></ul>
</div>

      <div class="footer-col footer-col-3">
        <p>记录、分享。</p>
      </div>
    </div>

  </div>

</footer>
</body>

</html>
